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ABSTRACT OR SUPPORTING INFORMATION 


Abstract 

This paper describes a methodology to efficiently construct feasible systems then modify feasible systems to meet successive goals 
by selecting from alternative components; a problem recognized to be n-p complete. The methodology provides a means to catalog 
and model alternative components. A presented system modeling structure is robust enough to model a wide variety of systems and 
provides a means to compare and evaluate alternative systems. These models act as input to a methodology for selecting alternative 
components to construct feasible systems and modify feasible systems to meet design goals and objectives. The presented 
algorithm’s ability to find a restricted solution, as defined by a unique set of requirements, is demonstrated against an exhaustive 
search of a sample of proposed shuttle modifications. The utility of the algorithm is demonstrated by comparing results from the 
algorithm with results from three NASA shuttle evolution studies using their value systems and assumptions. 
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A Goal-Seeking Strategy for Constructing Systems 
From Alternative Components 


Introduction 

Designing large-scale systems often involves making selections from multitudes of alternative 
components, subsystems, and configurations. Each potential selection must be weighed against 
several design constraints, goals, and objectives. Many selections involve trading a gain in one 
system parameter against a loss in another. For example, in one design study there were over 800 
modifications proposed for the Space Shuttle fleet [NASA, (22)]. Each of the proposed 
modifications affects system objectives for reducing weight, improving safety, increasing 
reliability, reducing operating costs, etc. A selection that improves safety may increase weight. A 
selection that increases reliability may increase operating costs. In addition, each modification 
consumes limited system resources such as power or budget. 

In the Shuttle illustration, each alternative modification interacts across several system 
parameters. These parameters may be described by discontinuous functions, or not be tractable 
mathematically and may require simulations or look-up tables. To date, the only proven 
optimization method for this type of problem is an exhaustive search of all feasible combinations 
[Bascaran, et al., (2)]. For the Shuttle, no alternative or set of alternatives is obviously dominant 
or infeasible and few of the modifications are mutually exclusive. As such, each combination of 
modifications must be considered as a separate competing system. This results in over 2 800 ( 1 0 240 ) 
component combinations to be evaluated If one million combinations were evaluated each 
second, it would take 10 227 years to perform this exhaustive search. 

Selecting components against multiple criteria is common in engineering design. However, most 
system optimization theory focuses on varying design and performance parameters of predefined 
systems with fixed configurations. Parameters are varied either to minimize deviation from some 
ideal, or to maximize an objective function Comparatively little work concerns defining initial 
system configurations or finding alternative configurations that offer improved performance. 

The methodology, described in this paper, is designed to construct a feasible system by selecting 
components from large inventories of component types. It then modifies a feasible system (again 
by adding or removing components) to meet successive performance goals. When all goals are 
met, it modifies a system to enhance system performance as measured by an objective function. It 
is designed to do this in far fewer iterations than exhaustive search would require. 


Background Concepts 

To understand the methodology described in this paper, it is necessary to have a common 
understanding of certain definitions and for using “distance” to compare competing systems. For 
the purposes of this paper, a component is a nondivisible element that may be added to or 
removed from a system A component has quantifiable and fixed properties. A system is a fixed 
set of components having well-defined boundaries and measurable effects on its environment. 
Parameters are single-term variables used to measure a single system or component effect (e.g. 
weight, length, cost). Any mathematical combination of parameters required to define some 
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system aspect defines a new parameter (e g. volume). A constraint is any mathematical limitation 
(e.g., >, <, =) placed on a system parameter value. 

A feasible system has parameter values that meet all imposed system constraints. A goal is a more 
stringent limitation or set of limitations that, when met, increases the perceived system value. An 
objective is a parameter constructed such that it describes a direction of increasing system value. 

In a fictional example, a state may require that cars sold within that state have a minimum mileage 
standard of 25mpg; this forms a manufacturer’s feasibility constraint. A tax credit is given as 
incentive for cars meeting a 30-mpg standard; this defines a goal. The manufacturer knows that 
the higher a car’s mileage is, the greater their sales will be; this defines an objective. 

“Distance,” as used to compare competing systems, was first developed in formulating the 
theories and arithmetic of “fuzzy logic.” Distance measures the divergence of two fuzzy numbers, 
or the degree of membership of a fuzzy number in a fuzzy set [Kaufman, Gupta, (17)]. As used 
for system comparisons, distance becomes the degree of divergence of a system from some ideal. 
The system closest to the ideal is the preferred system [Zeleny, (33)]. 


Model Construction 

To illustrate the component selection methodology, eight modifications were selected from 855 
proposed modifications in the “Shuttle Evolution Data Base”[22], For these eight, quantitative 
data have been tabulated for design, development, test, and evaluation (DDTE) cost, delta cost, 
weight delta, and Crit l’s (see Table 1). DDTE Cost is the total expenditure required to research, 
design, and implement a proposed modification for four Shuttles. Delta (A) Cost refers to the 
total change in annual Shuttle fleet operating costs the change would effect. Weight A can 
represent a change in Shuttle weight or performance; for example a 10-pound reduction in Shuttle 
weight equates to a 10-pound payload performance increase. Crit 1, or Criticality 1, items are 
Shuttle components whose failure will result in the loss of vehicle and crew. The “Crit 1” number 
is the total quantity of such items removed from or added to the Shuttle by that modification; 
removing Crit 1 items results in a safer vehicle. 


Table 1: Alternative Component (Modification) Listing 


Alternatives 

DDTE 
Cost $ 

A Cost $ 

Weight 

A#’s 

Critl’s 

C] 

3100000 

-1000 

-200 

-2 

c 2 

1.00* 1 0 s 

-100000 

-2200 

0 

C3 

2.60*10* 

-400000 

50 

0 

Ca 

6.50* 10 7 

-500000 

-288 

-6 

Cs 

4.93* 10 7 

-2000000 

200 

-13 

C 6 

2.25*10* 

-3000000 

-3500 

-37 

Cl 

1 .00* 10 7 

0 

-8000 

0 

C% 

1.25*10* 

0 

-500 

-44 
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For this example, feasibility and goal constraints will later be placed on DDTE Cost, A Cost, 
Weight A, and Crit 1 ’s. As such, each of the four data categories forms a system parameter as 
listed in Table 2. Each of these parameters corresponds to an axis in a four-dimension n-space. 

Table 2: Parameter Definitions 

a:, = DDTE Cost 

*2 = A Cost, annual operations cost change 
*3 = Weight A 

x 4 = Crit 1 ’s, change in quantity of Crit 1 items 

Given defined parameters, each alternative component type c k is described by an ordered list, also 
designated c k . These lists are similar to catalog descriptions that list product characteristics that 
would interest a customer. Each position c k j in the list contains the corresponding parameter 
value for a component of that type. Table 3 lists these component definitions. Parameter values 
for components are considered fixed properties. Components are vectorally additive only within 
the definition of the system vectors defined later. 

Corresponding to a warehouse, the set of available alternative components (inventory) is 
designated as 

I={iiCi i 2 c 2 i 3 c 3 i 4 c 4 i 5 c 5 i 6 c 6 i 7 c 7 i 8 c 8 }, 1) 

where i k = the quantity of type “k” components in the inventory, 

= {0, 1 }, for this example. 

For this example, a single component of each type is initially available, 

I 0 ={l(c 1 ) l(c 2 ) l(c 3 ) l(c 4 ) l(c 5 ) l(c 6 ) l(c 7 ) 1(c)}. 

Generally, any i* could contain any positive integer number of components. 

Table 3: Component Definitions 

C k =[.*i .X 2 .V 3 X 4 ] 2) 

3100000 -1000 - 200 -2 

1.00 £8 -100000 - 2200 0 

2.60E8 -400000 50 0 

6.50E7 -500000 -288 -6 

4.93ZT7 - 2000000 200 -13 

2.25ZT8 -3000000 -3500 -37 

1.00E7 0 - 8000 0 

1.25E8 0 -500 -44 



3 




Systems are defined as a set of components An initial system S 0 (i.e. before starting the 
algorithm) may be empty or have baselined components that the algorithm may remove The 
Shuttle has thousands of existing components However, in this example these components are 
not removable and are not included in S. If a proposed modification could potentially replace an 
existing component, then that existing component would be included in the initial system set. For 
this example, the system set is represented by 

S = {sjCi S2C2 S3C3 S4C4 S5C5 S6C6 S7C7 SgCg } , 3 ) 

where Sk = 1 if c k is included in the system, 

Sk = 0 if Ck is not included in the system. 

In the initial So, all Sk equal 0; the system set is empty. 


To compare performances of competing alternative systems, each system is plotted in 4-space 
based on its performance parameter values. A system vector is constructed to aggregate the 
effects of components included within a system. In this example, each system parameter is an 
additive function resulting in a system vector definition 


8 8 8 8 

SSk^kj £SkCk,2 Ss>k c k,3 zLsk^k.4 
_k=l k=l k=l k=l 



= s 


where Ck,i = the i-th property value for component type c k .. 
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For this example, constraints defining feasibility were selected for each of the four defined 
parameters and are listed in Table 4. These values were chosen only for their value in 
demonstrating the methodology. Together these constraints define a region of system feasibility 
in n-space bounded by orthogonal planes. This volume defining feasibility is designated F. 


Table 4: Feasibility Constraints 


-V, < 3 . 00 * 1 0 8 

x 2 < 2 . 50 * 10 6 

Xi < -8.00*1 0 3 

* 4 < - 5 . 00 * 10 ' 


A goal is also a set of bounded parameters defining a region in n-space and is designated G. To 
meet the goal defined for this example, the system must meet all conditions of feasibility at a 
lower DDTE cost as described in Table 5. 


Table 5: Goal Definition 


xi < 2 . 50 * 10 * 

*2 < 2 . 50 * 10 6 

*3 < -8.00* 10 3 

x 4 < -5.00*10' 
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Method Illustration: Feasibility Search, Hill Climbing 

The system construction algorithm starts with an inventory of possible components I 0 and an 
initial system So. First, components are removed from I 0 and added to So, constructing sequential 
systems whose corresponding vectors s gradually terminate closer to and finally within F. If 
adding components fails to result in a feasible system, components within S may be replaced with 
components from I. After a feasible system has been constructed, an attempt is made to modify it 
to meet the requirements of successive goals. 

The first step in the system construction process is to establish an initial baseline system. Table 6 
summarizes the system state prior to beginning the search algorithm. The initial baseline system 
So contains no components, all s k = 0. The baseline system vector s 0 , coincides with the origin, 
each s, = 0. The “F” column lists the parameter values of the feasible volume F. The distance d 
and the shadow distance d incorporate scaling factors, Ji. The scaling factors are used to 
compensate for variations of parameter measurement. For example, “Crit l’s” are measured in 
the tens and “DDTE Cost” is measured in the tens of millions. These variations in scale do not 
affect system feasibility determination However, they may affect algorithm performance. In this 
example, if scaling factors were not used, initial component selections would focus on reduction 
in DDTE cost because of its much higher value Crit l’s would be ignored until DDTE Cost was 
very close to or within feasibility limits. Large-scale differences most often result in much larger 
numbers of algorithm iterations, and increase the chances of failing to reach feasibility. 

Table 6: Initial System State, Iteration 0 

S 0 = {0(c,) 0(c 2 ) 0(c 3 ) 0(c 4 ) (>(c 5 ) 0(c 6 ) 0(c 7 ) 0(c 8 )} 


d = 1.732 
d = 1.000 


Xi 

Si 

F, 

d, 

d, 

1 DDTE Cost 

0 

300M 

0 

300M 

2 A Cost 

0 

-2.5M 

2.5M 

0 

3 Weight A 

0 

-8000 

8000 

0 

4 Crit l’s 

0 

-50 

50 

0 


s • - F: if S-. > Fs Fj - s { il dj - 0 

d: = d. = „ , 

|_ 0 otherwise “* 0 otherwise 

<!,) 2 

Figure 1 depicts a three-dimensional n-space and includes axes for the system parameters that are 

not within the feasible limits: jc 2 , jc 3 , jc 4 . The point in F closest to s is [ 1 ,- 1 1 ], normalized 

using the scaling factors 7 2 through ,? 4 . Since .vj is the only system parameter within feasibility 
limits, d = 1, normalized using 7], 


5 ) 

6 ) 
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In the first step of the algorithm, each alternative component c k in I 0 is successively incorporated 
into the baseline system S 0 to form alternative systems, S 0 (+k> = (So c k ). Since the current system 
baseline So = { } (before iteration 1), in iteration 1 each alternative system So<+k) consists of a 
single component c k . For each So<+t), a system vector So<+k) is determined. Using these vectors, the 
distance do<+ k ), from s 0 (+k)to F, and the shadow distance do^k) are calculated, as listed in Table 7, 
for each S 0 <+k)- Values associated with (So u c 6 ), the new root system, are listed below the 
tabulation for all systems. 




(_,- 1,0,0) 


- I - 


s =(_,0,0,0) 


3? 


(_.o,o,-iy. 




Figure 1: System Construction, Iteration 0 


Table 7: System Construction, Iteration 1 


7a: Results of Iteration 


So(+k) 

Sk 

doc+k) 

do"do(+k) 

do(+k) 

(do(+k)) _ do 

RATIO 

So (+6) 

l 

0.620 

1.112 

0.320 

-0.680 

0.432 

So(+5) 

0 

1.280 

0.452 

0.836 

-0 164 

0.288 

So(+7) 

0 

1.414 

0.318 

0.967 

- 0.033 

0.285 

So(+i) 

0 

1.695 

0.038 

0.990 

- 0.010 

0.027 

So(+4) 

0 

1.531 

0.201 

0.783 

- 0.217 

- 0.016 

S()(+8) 

0 

1.376 

0.356 

0.583 

- 0.417 

- 0.061 

So(+ 2 ) 

0 

1.564 

0.168 

0.667 

- 0.333 

- 0.166 

So<+3) 

0 

1.649 

0.083 

0.133 

- 0.867 

- 0.783 


Note: “(+k)” refers to the added component, e.g. S[*. 6 ) => c 6 has been added to 
the baseline system So. 
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RATIO — (do - do(+k>+)/(do - do(+k)), if (do - do<+k)) > 0 and (do - do(+k>) ^ 1 
= (d 0 - d 0 (+k)+)+(d 0 - d 0(+ k)), if (d n - d 0(+ k))>0 and (d 0 - d 0(+ k>) < 1 
= 0 otherwise 

Note: The change in defining RATIO, when (d 0 - do (+ k)) < 1, avoids distortingly high RATIO 
values because of very' small denominators. In those rare cases when resource is gained 
(do(+k> > do), negative RATIO is avoided. 


7b: Parameter Values for System (So c 6 ) 


*1 

Si 

F, 

d, 

d, 

1 DDTE Cost 

225M 

300M 

0 

75M 

2 A Cost 

-3M 

-2.5M 

0 

0.5M 

3 Weight A 

-3500 

-8000 

4500 

0 

4 Crit l’s 

-37 

-50 

13 

0 


7c: Comparison of d and d for baseline system and modified system 


So u c 6 

So 

d = 0.620, d = 0.320 

d = 1.732, d = 1.000 


Figure 2 shows the locations of the vectors resulting from alternative systems So(+n through So<+8), 
the baseline system vector So, and the feasible volume F. The vector for So(+6) terminated closest 
to F, and resulted in the highest RATIO. For the next iteration, S 0 (+6) becomes the new baseline 
system S 0 . 



Figure 2: System Construction, Iteration 1 


In iteration 2 (Table 8), the process of iteration 1 repeats. Alternative systems are formed 
combining c 6 successively with the remaining alternative components. In this iteration, the 
combination of c 6 and c 5 results in the largest RATIO of decrease in d to decrease in d. 
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Table 8: System Construction, Iteration 2 


8a: Results of Iteration 



Sk 

do(+k) 

do“do(+k) 




S 0 (+6) 

l 






S<)(+5) 

l 

0.588 

0.032 

1.004 

0.684 

0.716 

So(+7) 

0 

0.260 

0.360 

0.528 

0.207 

0.567 

So(+8) 

0 

0.527 

0.093 

0.651 

0.331 

0.424 

So(+4) 

0 

0.545 

0.075 

0.401 

0.081 

0.156 

So(+ 2 ) 

0 

0.396 

0.223 

0.240 

-0.080 

0.143 

Soc+ 1 ) 

0 

0.581 

0.039 

0.312 

-0.008 

-0.031 

So(+3) 

0 

0.878 

-0.259 

0.360 

0.040 

0.000 


8b: Parameter Values for System (So ^ Cs) 



Si 

F, 

d, 

d, 

1 DDTE Cost 

274.3M 

300M 

0 

25.7M 

2 A Cost 

-5M 

-2.5M 

0 

2.5M 

3 Weight A 

-3300 

-8000 

4700 

0 

4 Crit l’s 

-50 

-50 

0 

0 


8c: Comparison of d and d for baseline system and modified system 


S 0 v_j c 5 

So 

d = 0.588, d = 1.004 

d = 0.620, d = 0.320 


Figure 3 depicts the seven alternative system vectors and the baseline system vector. Since all 
parameters are algebraically additive, the alternative system vectors have been translated by 
simple vector addition to s 0 - Notice that several vectors are closer to F than so<+5) However s 0 <+5) 
had the highest RATIO of decrease in d to decrease in d. 
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In the third iteration (Table 9), c 7 is the first to be considered for addition to the baseline system. 
The alternative system formed when c 7 is added meets all conditions for feasibility. At this point, 
goals are not a consideration and this stage of the algorithm ends. No other alternative 
components are evaluated. The feasible baseline system consists of S 0 = {c 6 , c 5 , c 7 }. This set of 
components forms the baseline system for the start of the goal search. 

Table 9: System Construction, Iteration 3 


9a: Results of Iteration 


So(+k) 

Sk 

do(+k) 

do-do(+k) 

do(+k) 

(do(+k))-do 

RATIO 

S 0 (+6) 

1 






S<)(+5) 

1 






So<+7) 

1 

0 

0.588 

0.528 

-0.476 

0.111 


9b: Parameter Values for System (S 0 vj c 7 ) 


-v, F, d, d, 

1 DDTE Cost 

284.3M 

300M 

0 

15.7M 

2 A Cost 

-5M 

-2.5M 

0 

2.5M 

3 Weight A 

-1 1300 

-8000 

0 

33000 

4 Crit 1 's 

-50 

-50 

0 

0 


9c: Comparison of d and d for baseline system and modified system 


So CJ c 7 

So 

d = 0.000, d = 1.083 

d = 0.588, d = 1.004 


Method Illustration: Goal Search, Backtracking 

Table 10 contains the starting conditions for the goal search portion of the algorithm. The budget 
for DDTE cost is reduced from $300M to $250M. All other conditions set for this goal are the 
same as those set for feasibility. In this initial goal search iteration, only the baseline system is 
evaluated in order to determine the new baseline d and d. 

Table 10: System Modification, Iteration 4 

S 0 = {0(o, ) 0(c 2 ) 0(c 3 ) (>(c 4 ) l(c 5 ) l(c 6 ) l(c 7 ) 0(c 8 )} 


Xi 

Si 

G, 

D 

d, 

1 DDTE Cost 

284. 3M 

300M 

0 

15.7M 

2 A Cost 

-5M 

-2.5M 

0 

2.5M 

3 Weight A 

-11300 

-8000 

0 

33000 

4 Crit 1 ’s 

-50 

-50 

0 

0 


d = 0.1 14, d = 1.082 
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In the fifth iteration (Table 1 1), the remaining components are sequentially added to So and the 
distance to the goal G is determined for each None of the alternative component incorporations 
resulted in a reduction of d as compared to the baseline system. This would be expected since the 
addition of components increases system DDTE Cost. The next iteration begins using a 
backtracking strategy in an attempt to find an alternative basis system that will eventually allow s 
to terminate closer to, or within, G. 

In this backtracking strategy, successive components c k are removed from So to form alternative 
root systems So(-k> Each of these alternative root systems is tested using the hill-climbing 
technique used to reach F. The alternative root, with any added components, that terminates 
closest to G (or F if appropriate) becomes the new basis system. 


Table 11: System Modification, Iteration 5 



Sk 

d<X+k) 

dcrdo(+k) 

do(+k) 

(do(+k)) _ do 

RATIO 

S|}(+6) 

l 






So(+5) 

l 






Sq(+7) 

l 






So(+8) 

0 

0.531 

-0.417 

1.414 

0.332 

0 

So(+4) 

0 

0.331 

-0.217 

1.287 

0.205 

0 

S()(+2) 

0 

0.448 

-0.333 

1.247 

0.165 

0 

S()(+l) 

0 

0.125 

-0.010 

1.093 

0.011 

0 

So(+3) 

0 

0.981 

-0.867 

1.229 

0.147 

0 


Iteration six (Table 12) establishes an alternative root system So(-6) by removing c 6 from the 
current So. That alternative system is then tested for goal achievement. In this case, d is greater 
than 0; the system does not meet goal conditions. Removal of c 6 has reduced DDTE Costs, but 
all other parameters moved away from the goal The net effect is a distance increase. 

Table 12: System Modification, Iteration 6 
12a: System at Iteration 6 

S = {0(c, ) 0(c 2 ) 0(c 3 ) 0(c 4 ) l(c 5 ) ()(c 6 ) l(c 7 ) 0(c 8 )} 


12b: Parameter Values for System (Sp \ c 6 ) 



■Si 

F, 

d, 

d, 

1 DDTE Cost 

59.3M 

250M 

0 

190.7M 

2 A Cost 

-2M 

-2.5M 

AMO 

0 

3 Weight A 

-7800 

-8000 

2000 

0 

4 Crit l’s 

-13 

-50 

370 

0 


1 2c: Comparison of d and d for baseline system and modified sys tem 


S 0 \ £*6 

So 

d = 0.767, d = 0.636 

d = 0.1 14, d = 1.082 
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The alternative root So(-6) is expanded using the hill-climbing technique. Each of the previously 
unused components is successively added to So ( .6). The resulting systems are then evaluated. In 
Table 13, Cg resulted in the largest RATIO and is added to So<-6) This system provides a reduction 
in d as compared to So(-6> and another iteration will be made. 

Table 13: System Modification, Iteration 7 


13a: Results of Iteration 


So(+k) 

Sk 

do(+k) 

do-d 0 (+k) 

do(+k) 

(do(+k))"do 

RATIO 

So(-6) 

0 






So(+5) 

l 






S(I(+7) 

l 






So(+8) 

l 

0.200 

0.567 

0.263 

-0.373 

0.194 

S()(+l) 

0 

0.728 

0.039 

0.625 

-0.010 

0.029 

So(+3) 

0 

0.777 

-0.010 

0.000 

-0.636 

0.000 

So(+4) 

0 

0.620 

0.147 

0.419 

-0.217 

-0.070 

So(+ 2 ) 

0 

0.757 

0.010 

0.392 

-0.243 

-0.234 


13b: Parameter Values for System (Sp^ Cg) 


*i 

Sx 

F, 

d, 

d, 

1 DDTE Cost 

184. 3M 

250M 

0 

65. 7M 

2 A Cost 

-2M 

-2.5M 

0.5M 

0 

3 Weight A 

-8300 

-8000 

0 

300 

4 Crit l’s 

-57 

l 

L/\ 

o 

0 

7 


13c: Comparison of d and d 


So<-6) ^ c 8 

So(-6) 

d = 0.200, d = 0.263 

d = 0.767, d = 0.636 


In the next iteration (Table 14), the baseline alternative root system consists of components c$, Cj, 
and eg. Again the hill-climbing technique is employed and successive components are added to 
the alternative root. Upon the addition of c 4 , all goal conditions are met and the algorithm ends 
having met a satisfying solution. The final baseline system, meeting all conditions of F and the G, 
is So = {c 5 , c 7 , eg, c 4 }. 

If this iteration had not been successful, hill climbing would have been repeated until no decrease 
in d was realized. These results would be stored for the alternative root So<-6). The procedure 
would then be repeated for each of the remaining system components. If none of these alternative 
roots met the goal conditions, the alternative resulting in the greatest reduction in d would 
become the new baseline system. The process would then repeat. If no alternative root results in 
a reduction of d, the algorithm terminates. 
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Table 14: System Modification, Iteration 8 


14a: Results of Iteration 


So(+k) 

Sk 

do(+k) 

dn-do(+k) 

do(+k) 

(do(+k))-do 

RATIO 

So (-6) 

0 






So(+5) 

l 






So (+ 7 ) 

i 






So(+8) 

l 






So(+l) 

0 

0.200 

0.000 

0.283 

0.020 

0.020 

So(+3) 

0 

0.649 

-0.449 

0.143 

-0.119 

0.000 

So(+4) 

1 

0 

0.200 

0.419 

0.157 

0.357 

So(+ 2 ) 








14b: Parameter Values for System (S w .6) u c g ^ c 4 ) 




F, 

d, 

d, 

1 DDTE Cost 

249.3M 

250M 

0 

0.7M 

2 A Cost 

-2.5M 

-2.5M 

0M 

0 

3 Weight A 

-8588 

-8000 

0 

588 

4 Crit l’s 

-63 

-50 

0 
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14c: Comparison of d and d 


So(-6) rj c 4 

Sq(-6) ^ 

d = 0, d = 0.270 

d = 0.200, d = 0.263 


Method Comparison With Previous Study Results 

A variety of Shuttle evolution studies have been completed over the years. Of these, three used 
roughly the same database of possible modifications that the previous illustration was based on. 
The input and results of these three studies are used to validate results obtained using the method 
described in this paper. 

The first study used a PC-based implementation of the analytic hierarchy process (AHP) decision 
analysis method [NASA, (21)]. Safety, Mission, Performance, Turnaround, and Cost were given 
relative weightings used to score Shuttle enhancement alternatives. These enhancements were 
then ranked and 25 were given a 1, 2, or 3 priority (Table 15). 

To compare this first study’s results with those using this paper’s alternative method, the 
Weighted Total score, the Safety score and the Cost for all Priority 1 items were totaled. These 
results were then input as feasibility limits into the PC application of this paper’s method. After 
1 1 iterations, the minimum limits placed on Total and Safety were surpassed, while staying below 
the Cost limit. After the initial minimum conditions were met, the goal for Total was successively 
raised until it could not be met within the fund limit. This paper’s method took 66 iterations and 
less than 1650 system calculations (<1 hour, 33MHZ 486). Exhaustive search would have 
required 33,554,432 system calculations. These results are compared in Table 16. 
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Table 15: Study One Results 


Priority One Items Priority Three Items 

Alternative 

ABCDEFG ABCD 

Weighted Total 

482 478 475 433 419 390 386 99 96 95 91 

Safety 

500 500 500 500 500 100 100 100 

Mission 

450 450 

Payload 

-6 -20 -18 -34 -9 

Capability 


Turnaround 

100 

Life 

3 

Cost 

-15 -2 -20 -17 -63 -60 -80 -1 -4 -5 -9 

Payback 

9 

Cost $M 

14.6 2.4 20 16.7 63 59.9 80 1.3 4.2 4.8 9.4 


Priority Two Items 

Alternative 

A 

B 

c 

D 

E 

F 

G 

H 

1 

J 

K 

D 



Weighted Total 

267 

250 

247 

246 

245 

240 

237 

237 

230 

227 

225 

211 

192 

182 

Safety 

250 

250 

250 

250 

250 


250 

250 


250 

250 

250 


250 

Mission 






450 



150 






Payload 

-16 






-2 





-39 

210 


Capability 

20 






30 






25 


Turnaround 






-84 

-32 







-63 

Life 

80 

Cost 

-19 

0 

-3 

-4 

-5 

-126 

9 

-13 

-12 

-23 

-25 

-25 

-18 

-5 

Payback 

4 
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Cost $M 

19 

0.0 

2.8 

3.7 

4.5 

126 

9.2 

12.5 

12 

23.0 

25 

24.7 

18 

4.9 


Alternatives in boldface are those selected using the method outlined in this paper. 


Table 16: Study One Results Comparison 



Study One 

Alternative Approach 

Total (points) 

3500 

5515 

Safety (points) 

2500 

4800 

Cost ($M) 

257 

253 


Another internal NASA study using an alternative procedure ranked and scored 32 alternatives 
(Table 17) [ECON, (7)]. Scores or ratings were assigned for Nonrecurring Cost (NRC), Cost 
Savings, and Safety. These were combined into a total rank score. For the Safety category, each 
alternative was rated as having a high, medium, or low impact. For alternatives with similar utility 
scores, the alternative with a higher Safety rating was given preference. 

To compare the second study results with those achievable by this paper’s method, Nonrecurring 
Cost in $M (NRC), Savings ($M), and Safety were totaled for the top five ranked candidates. To 
quantify Safety, high was given a score of 3,000, medium 200, and low 1; these scores were 
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chosen to ensure that any alternative set of components would have at least as many high- and 
medium-rated components as the second study’s top 5. These resulting sums were input for 
feasibility limits. After 5 iterations, Safety and Savings were exceeded at 2/3 the NRC. The limit 
for savings was increased in increments of 1 00 until no additional benefit could be achieved at the 
original NRC. The results are tabulated in Table 18. 

This paper’s method took 1 12 iterations and less than 3,584 system calculations. Exhaustive 
search would have required 4,294,967,296 system calculations. 


Table 17: Study Two Results 


Alter- 

native 

NRC 

$M 

$M 

Save 

Safety 

Comp. 

Score 

17 

21 

3 

low 

27 

18 

90 

43 

low 

26 

19 

90 

3 

low 

26 

20 

79 

38 

low 

23 

21 

68 

3 

low 

20 

22 

65 

25 

low 

19 

23 

58 

22 

Med. 

17 

24 

55 

0 

low 

16 

25 

39 

0 

low 

11 

26 

25 

-7 

Med. 

11 

27 

33 

13 

low 

9 

28 

28 

1 

low 

8 

29 

23 

6 

Med. 

7 

30 

19 

-9048 

low 

6 

31 

4 

6 

low 

1 

32 

3 

6 

low 

1 


Alter- 

native 

NRC 

$M 

$M 

Save 

Safety 

Comp. 

Score 

1 

296 

423 

High 

100 

2 

334 

122 

Med. 

96 I 

3 

300 

-231 

High 

87 

4 

135 

62 

High 

81 

5 

170 

-15 

low 

57 

6 

168 

119 

High 

49 

7 

28 

48 

low 

46 

8 

155 

-18 

low 

45 

9 

87 

119 

High 

42 

10 

143 

63 

Med. 

42 

11 

141 

0 

Med. 

41 

12 

33 

-11 

Med. 

32 

13 

101 

8 

low 

29 

14 

23 

3 

low 

28 

15 

40 

162 

Med. 

28 

16 

88 

-18 

low 

27 


Bold indicates final set of modifications selected by alternative method. 


Table 18: Study Two Results Comparison 



Study Two 

Alternative Approach 

NRC, $M 

1,235 

1,217 

Savings, $M 

361 

1,123 

Safety 

9,201 

12,808 

i.e. 

3 -highs 

4-highs 


1 -medium 

4 -mediums 


1 -low 

8 -lows 


The third study was based on an exhaustive survey resulting in a list of 800 proposed Shuttle 
modifications [NASA, (22)]. The list included performance scores for DDTE costs, Recurring 
Costs, Safety, and Payload Capacity. Of the 800 alternatives, only 210 included DDTE costs and 
positive scores for at least one performance measurement. 

The 20 highest scoring alternatives were aggregated to set the minimum standards for the test. 
The algorithm met Recurring Cost and Capacity scores, and exceeded the Safety and Total sums 
at half the DDTE cost. After meeting the minimum conditions, the goal values were raised by 
approximately a third with the same cost limit. These limits were also met (Table 19). 
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Table 19: Study Three Results Comparison 



Study Three 

Alternative Approach 

DDTE, ($M) 

8,183 

6,658 

Safety (points) 

94 

182 

Recurring Cost (points) 

117 

150 

Capacity (points) 

105 

150 

Total (points) 

316 

471 


Running overnight on a 33MHZ 486, the algorithm took 83 iterations and less than 17,430 system 
calculations. Exhaustive search would have required 1 .65* 1 0 63 system calculations. 


Summary 

The outlined methodology provides a robust means to catalog and model alternative components 
that are available for constructing a given system and to model that system With these models, 
the methodology may be used to select components that together form a feasible system. Once a 
feasible system has been constructed, the methodology is capable of modifying that system to 
meet successive goals or to increase the value of an objective function. 

In verification trials, the developed methodology was found capable of duplicating the results of 
an exhaustive search in far fewer iterations. When applied to the data used in three Shuttle 
evolution studies, the developed methodology produced component sets exceeding the 
performance of study recommendations at the same or a lower budget. 

The only proven general method to produce optimality for component search problems is 
exhaustive search. Exhaustive search is usually impossible for large problems due to the number 
of possible component combinations Although the described method is not an optimization 
method, it has proven useful in producing good and often close to optimal results. The described 
method provides a general and efficient means to evaluate and select from large numbers of 
alternative components in order to construct systems. Even at its current stage of development, it 
has proven to be a capable alternative to methods currently used for this type of problem. 
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